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ABSTRACT — • . 

A^seninar is described in vhich systeiatic 
observation techniques, were applied to lecturer, training to take the . 
supervisory element in Bicroteaching acre objective. The systea 
provided a highly structured technique that the lecturers could use 
to systematise their discussion of aicrolectures or real lecturjes, * 

'either recorded on, tape or live. Three""paraaeter§ irere ••pbs.erved**: 
non-verbal; ver&al, arid content aspects of lecturing. These . " 
Categories vere used after the trainees learned the systea and could 
reliably code a lecture into these coaponents. This inforaatipn was 
then used by the trainer and trainee as data in analyzing a lecture. 
A saall scale study is repor^^ted in vhich the attitude of the trainees 
to each ^ fo'ur treatments (systeaatic observation vs. conventional 
feedback and audio vs.. closed circuit television, augaented feedback) 
was measured* .Mo significant difference was found between groups. A \ 
discussion of the reliability and validity of the systea is appended. 

rIDAG) ^ - . ^ . - 




♦ Docuaents acquired by ERIC include aany inforaal* unpublished *♦ 

♦ materials not available f roa othe? sources. ERIC aakes every effort ♦ 

♦ to obtciin the best copy available. Nevertheless, iteas of aarginal ♦ 

♦ reproducibility are often encountered and this affects the quality ♦ 
of the microfiche and hardcopy . reproductions ERIC aakes available * 

. ♦ .via the ERIC Dotruaent Reproduction Service (EDRS) . EDRS is not ♦ 

♦ responsible for the quality of the original docuaent. Reproductions * 

♦ supplied by EDRS are the best that can be aade froa the original. * 



"•*'j>5^A«TMENT OF HEALTH. 
C0UCATI0N4WELFAHE. 
' NATION AL tMSTiTUT^ 

COUCATtON 



DOCUMENT HAS-fBEEN REPRO. 



The-^ Systematic Observation of Lecturing 
its use in training and. research. 



Ray McAleese 

0 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED. BY 

Ray McAleeseo 



TO. THE EDUCATIONAL RESOURCES 
* INFORMATION CENTER {ERIC) AND 
USERS OF THE €RIC SYSTEM/' 



DeparfcRicnt of Education, 

King's College, ' ^ , 

University of Aberdeen. 

, Sepbember, 1973 ^ - 

A paper presented at the SSRC seminar on Classroom Research 
held in the University of Leicester, 26-28 September, 1973. 




My contilbutionpxfetr-^ seminar vrLll be conceined with the 
work that I hav^mdertafcen in the University of Aberdeen. 
Conventional micro teaching techniques have be^n successfully, 
^employed. In this work, ^however, I have attempted to see, if 
r couljl apply systematic.obs^ervation techniques to 
lecturer training.- , , , 

In Dublin, * I upsd systematic observation (FIAC categories) 
to make more objective the supervisdi^- element in raicroteaching 
training. (McAleese, \m) I have, foymd 4at t^, mC:=.t3^i -.^f^ 
categories- provided an objr-^^— ^^->--*— f'^r. ^r^^r^ ^r.y^.^j 
sessions, 
appro £ 

^d.th I' decided to .concentrate ori ,l;(iotj^r|ng /diSa-ctic or 
monologue teaching) - af^^er l^'mbntKs.I have 'some hopes of 
increasing the coverage of the tegfinique to tutorials and 
practicals, but in the first instance ttie work has been 
■ concerned with, lecturing . This decision to concentrate oh 

lectures was taken primarily on grounds of convenience. The 

*» 

attraction of systemat^-c observation • to tutorials was out- 
"'■weighed by the fact that even in, 1973 lecturers still find 
most of their teaching timo/in formal lectures!. In practice,, 
lecturing is still, to those involved in training, the more , • 
important activity (a point which may not be Innovative, but 
reflects the aspect o£^he tvainers role concerned' vrlth- 
maintaining the status quo . ) Although initially I was hopeful 
.that Simon and Boycr (1970)- could give me some 'leads to 



2. 



4 \ 



^possible obs.erviition systems, this was not .possible^ I had 
.to start froia scratch in the con^straotion,, validafion and ' . 
' implementation of a new category system. Lecturing and 
-lecturers I discovered viere different iii several ways from 



either what is irreverently called^ » ping-pong" teacldng or from 

- — •> — — - — 

.any o.ther type of teaching analysed in Mirrors. \ 

Pirstly,> lecturing is basically non-interactive: it is 

monologue. Although this .may^or may not be desirable depending 
-on oner's viev/point, lecturing is not 66% teacher talfc (e.g. 

secpndary » ping-pong » ) but 100^^ teacher talfcl Therefore, any 

A 

system 'of interaction analysis had" to be 'discounted. 

Secondly, the teachers that I was^ to be dealing \-iith were 
by and large blissjTully ignorant or educational jargoni j?^ -Y6u 
might say that this is an advantage, but it does .^ose pi^blems 
when one thinks of using same system that %a:s^:aaon 
instructional concepts inbedded in educational phraseology,, 
e.g. Taba or Aschner and G^lagher. Gfaba^^ JL?yine and Freedman, 
19675 Aschner and Gallagher, 1?.65). , ^ * - - * 

Thirdly, I required a system that would--bi applicable to 
all the various disciplines in the university - from medicine 
to mathematics.' ^ A reqviirement that may need modification in 
' light of the . diversity in style shown by some of my colleagues 
• in various subjects that vie have looked aL. • 

It was decided that the system shou].d provide a technique'* 
that the* lecturers could use to systematise their discussions oL 
microlectures or real lectures either recorded on "tape or live. 
Using the Simon and Boyer (1970)^ classification^ T was- looking 
for the system that could be described by the following- 
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parameters • It can be:- , 

I'; Used on any srabaect matter ^ 

2. The sole Object of obselrvation is 'the le^cturer. 

— — ■ — — — . • — '■ 'i ' ' " " ~ ' 

. ^3,. .Orily-oneJLecturerlnoea-bel-.obseryjadlai^ one time. 

- It should be able to be used in research, training 

and evaluation,. , , , ' 

Data must be able to be collecjbed live, using one 
observer. - * ' * ^ 

* ,6. The coding 'unit should be time units. 
I decided using these parameters to » observe » the pedagogic 
structure of the lecture. I had the dioiceof looking at' 
whcit niight be called either the Non-verbal, the Verbal or the 
Content aspects of lecturing. Verbal ''structure was chosen in 
the first instance as it provided fewest** problems given the 
constraints of didactic teaching and the nature qX the 
lecturers* " ' ^ ^ - 

The analysis of verbal structure vjas also chosen as it 
provided what can be called a meta-language of supervasioh . 
As most of the loctu^i^r training in Aberdeen is involved with 
heterogeneous. groups -of lecturers, a problem is that one has 
'in the same course mathematicians, chemists^ geologists,, etc.^ 
The variety of discipline represeriteli is by and large an 
advantage, but ;:jJien it comes to discussing the individual 
•content of microlectures the trainer is presented v/ith 'some- 
thing, of a problem. The participants,, teaching their own 
subject, find that conparisons between lectures is also very 
difficult. As this is one of the key elements of the 

^ 

training - that is making peer comparisons, a system that 
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■ ■• • • . i. -j-^. " 


% 

1 

s 


alleviates this problem is to be preferred^ By analysing the ^ ' . 

• • • 

* content of lectures in terms of Teaching Points, Examples, - - 




Asides,, etc^ ijt is possible to" construct a very simple meta-language 




of instructi&fi* - . . • • ^ 


" * 1 


ft • ■ H " 

Figure 1 about ^here . 


c 


'p.o f^atepories^in Jleure 1 are' the vjords of thi's language and 




also thb-syijtem of observation, itself . The participants are 




able to communicate their value judgments about their^ lectures 




by using this language,. instead of using' mathematical. 




chemical or geological teniis. Taking this metaphor just a ; 




little further I hope' tha.t it may be possible to construct a 
' 'grammar of lecturing,' although this is not possible at- 
present..- The const^ction- of the gra-war can be thought of as ^ 
the research element of this ongoing study. ^ _ _ 

.Haying now desoribed-.hov; .I_ came to use the system and ^ - . 
having given you some idea of its application let me- say how _ ■ -. 
i have used the system in training an4 research, (a note on 




the reliability and validity of the system is contained in 




the Appendix to this paper). ' . . , " 
TRATHIKD * ' . , • 




' ' One reason for using a systematic observation system is to 

' / . • V ' 

give a_.high degree of structur^e to the supervisory session \Ln 




• the- microteaf;hing cycle. Normally when the trainees have 
viewed their microlecinire the supervisor adopts a non-directive 
role and u.^ing peer criticism to emphasise points - he 

"shapes" in a non-threatening way the behaviour of the trainee. . 
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Sessions' like this, no nattel* hovr good the supervisor ±Sj cm 
becoFie rather rambling and more non-directive than desiredl 
„ Systematise^ pbservation da^^ .a-focus^-to^ the-discussiofi and 

also provides data that describe the teaching performance in 
terms -that have unique meanings. Before the category system 
can be used the ^trainees must learn the system and tAis 
requires an initial training session that is devoted to^ shaping 
the trainees » behaviour in Vecognising and recording the - 
various categories. Figure 2 summarises this process. 

^ 

Figure 2 aboui here 

Having completed the training, the lecturers should^ be 
able to listen to a lecture and reliably cJde"?he content on 
to a Time-E5.ne, Display (TED). The TLD is used as it permits 

r 

quick inspection of the lecture and keeps the data, in its 
original seouence. Further, the TLD allows the 'trainees to 
look for what -vre are calling- the 'profile of the' lecture' . 
That is the number of "ups" and"doTOs", etc. that can- be seen 
in category changes. 

\ ' 

^ Figure 3 about here 

During the actual training sessions this infp_^ination is^ 
used by trainer and trainee as data. Tliat is, if the trainee 
wants to make a point aboitt a certain aspect of the lecture, he 
can refer to the TLD for an » objective ^ evaluation or r.ecord of 
the lecture at that point. He can^use the data to facilitate 
explanations of why any part of the lecture was less good than 
otlier parts. The trainee can refer back to a particular point 



in the lecture and find out v;hether the content analysis can^ help 
explain vniy lesture -seemed to go wrong. This type of 
an^sis begs for 'Norms' to be established for dd.ffereht types- 
of lecturing. It is this stage of the research, that we are now 
moving on to as we intend to undertake a study of lecturing in 
Aberdeen and apply these categories to a variety of lectures 
'in ah atteiijpt to establish some idea of »Noitos»*. This work 
has just -started and so* far vje have no usable resultsi We are 
undertaking a correl'ational study and we^ hope to look for * 

relationships betvjeen lecturer verbal behaviour, (teaqhS.ng , - 

^effectiveness?) and student .perforfaance.^ It is also our 
intention to take the correlational study further by "feStablish- 
ing what may lie causal relationships in experimental studies* 

• Let'' me conclude this contributioli by reporting on a^ small 
scale st\0y that I underj^ook earlier thib year in an attempt to 
investigate how this systematic observation was^ received by 
lecturers in tr^ning. I used a 2 x-2 design for -the 

independent variables. Variable A" Systematic observation 

vs. Conventional feedback. Variable B Audio only 

augmented feedback vs* CCTV augmented feedback. The dependent 
variai)le was the attitude of the trainees to each of the four 
treatments. Not a very) ' strong ' variable but one that we feel 
is important if this developmental work is ta succeed. Using 
multiple choice and^ open-ended questions for the dependent 
variable there was no significant difference between groups. 
A result that vyas in one sense not unexpected due to the crnidity 
of the dcpi^ndeiTt variable. However, other^^explmiatioris of' 
the non-Gignificant .diTXerences that are of more immediate 



•in"Ler.est to ijy \3ork v;oula be that there wa^s a large Hawtliprrie 
, effect on our twdnees . . However the open-ended questions 
indicated that the trainees did accept .this new type of 
training and ue are further encouraged by follow-up that the 
'lecturers can do themselves. I think that .this follo'w-up is 
particularly attractive as after the initial traiping is over, 
as the lecturer applies the categories to their own lecturing 
in vivo and attempts to find "behavioural e3q)lanations of 
successful or »disastrou5»^" lectures. This work is still in. 
•embryo. , 

My research, and development ''Kas'jusr^ it will 

I hope be possible in another year or sp^ to make a more- 
objective evaluation of its efficacy. At' present we hav^ 
still a lot of problems to solve. , Tor exajiiple, How^b* 
measure change in behaviour, ,how to make; statistical comparisons 
between ^profiles*.. Looking back to the early sixties I think 
that iiiis work Is in about the same position as the observation, 
systems were in about' 1962 or 1963. I airi^ however, hopeful 
of similarly successful temination' of itis gestation period!. ' 



The Reliability and Validity of the Categories- ' 

» — ' ' * * 

*1* The starting point of the system is the research r^ppfted by 

♦ 

Gage (1968).' In .this he suggested that one 'element o'f 
lecturing that might discriminate between effective -and 
ineffective lecturers was their ability to relate examples (Egs) 
to teaching points (Rules). . This «inil-eg» unit, he suggested, 
is found more in 'effective than in ineffective lecturers. K 
Rules (Toints) and Examples give the first two c£^begories of" 
the -system. • - 

2. The other categories were -devised by systematically 
observing lecturing and deciding on recognisable events' that 
make up the rest of this didactic teaching; (McAleese, 4972 )i 
In initial trials it vias found that lecturing could be veiy 
crudely di^/ided into ■ TEACHING- (Points (P),^; Ex^ples (E)) and 
MANAGB-IBHT . The% management section could be sulS-divided into 
two. categories, Asides,, that viere jjistructional in intent 
and Asides that viere nori-rinstructional, or useless (Au),..... . 

in relation to the teaching of the topic. The fifth category 



X V7au added as it' was decided to analyse the content using a 

tjme unit method of ssunpling and a dustbin categoiy was - 
' ' ' ' ' ** ,9 

required. ' 

3. Analj'sis of the sentence. structure of lectures indicated 

that on averages the time taken to say -one recognisable 

, "thought unit-» (Taba, 196*7) .... e.^g. Point, Example vras 

<• 

11.00 seconds* Using this piece of information, it was 

• - - \ 



decided to adopt a 6 second time unit for sampling, in training 
*at leasti - S6me 'other iv'ork using 3 second intervals has-been 
attempted as the shorter time unit causes fev/er decision^ 



ahout multiple coding, during any particular time \mit.* The 
shorter ^sampling is also more applicable to-rosear.ch and " 
validation.' The "Validity of this point is still to be 
established. If the meaft time a, thought unit is 

greater, than the length of coding' time unit, then there may * 
be problems in using convvaitional t-A statistics • j 

h* The five ^categories were applied on tv7o occasions to ^. 



samples of teaching, with the follox^ng results. ' Clearly, 
Examples are the most difJCicuXt to opera tionalise and 
therefore to observe. 



V 



Table 1 about here 



This gave a measure of the .Construct reliability or . Stability 
of the system. • . ^ 



5« A measure of .^the concurrent validity of the system was 
attempted by devising a 12 point rating -scale., instrument that- 
paralleled' the qbsdi^ation sy^ton and applyiiig this to 
16 samples of lecturing* A test-retest reliability cbeff-?-cient 
for the rating scale vjas e^stijnated^as 0.8 Qorrelations fob 
each of the categorie^vrith the rating instrument are found 
in Table 2. 



Table 2 about here 



6., Further and more detailed valid? tioij ol* the syfjtem is underaay:. 
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\ . ■ DEPAR-niEIT O^* EDUCATION ^ ' ^ 
.^--^ OIirVERSITY e? ABEI'J^EEI^ 



COIITS:iT a}:ai.ysis LECTURIMG (C.A-rL-.)- 



- -t - 



Glossary 

• A teachirtg^jjoint, part of an argtiment 
. • • • A * • 
oi; thesite. An item of ^infonnation, 

an oj^nion, a fact, a generalization, 

a :principle or conceDt, Lecturer »s 

own opinijjrt or- ducting another 

.authority. ' ' ^ . ^ 



Coding Unit 



.^uth 



POINT 



P--. Exaj!\ples, r.elated to' teaching points. 
Cpmparitors, a concrete example of a- 
teaohing point. • • 



example; 



Verbal behaviour of relevance to * 

the lecture but not intende^i as V^rt ^ 

of an argument or thesis*. ' 

References, objecti/es' or aims - / 

Relevant to the lecture or"* course of 

le^res. Summarising teaching ^ 

^polats made in other lectures. 

*• • * * , • 

*^ 

4 non-ins truciidondl aside'. • Comments 
of no direfft relevance to topic of 
leiSlure. Greetings', 'tension 
rejjeaserjs^^.directibns, ;iokes or 
irreXevSnS examples. " * ' ^ 



' ASIDE . 
(instructional) 



• ^ ASIDE 
(Non-instructional 
/useless)' 



Verbai;jbohaviour that is uncodable. 
Confusion or silence. Moh-verbal 
^ connsanication. Vfritin/i on , blackboard, 
using A-V aid,- Student Talk. ' ' * 



Symbol 




•Ai 



Au . 



, X' 



ERJC Figure 1 : Categories v 
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•DAY i 




* Trainiiv^ in Observation Systsrn 



Vicv: video tr^^e of lecuur in > lectur-^'.thc:a tre . 



2\ 
3. 


\Discus3 

•^The cacegories of the ob'seryatioa systein 




6. 


Listen. to c::an:Dles of cate;:xor.ios ^ 




• 


L;r.tc:!i to lecture (sar.e as 1), looking for exarapies of 


categories 


6. 


Discuss 




7 0^ 


Code transcription of lectures (same as 1) 
The Tir.e-iine display (TLD) 






Trans f-rr coding of lecture (7.) to TtD 




; la. 


San:pling of 'teciching behaviour (3 secortds; 6 seconds) 




11. 


Code TvDescript of lecture' (same' as I) at 6 second- intervals ' 


12, 


Transfer typescript (li.) to TLD 




13.. 


JS.ar*p;lihg at 6, second- intervals, (listen to timing) 




. 14". 


Listen CO lecture . (.same as ^ I) • • , 






Code- lecture at 'jS. second . intervals (same as 1. ) _ * 




re. 

' xt. 


Transfer lecture, (Ip.^ to TLD ' , 
Discuss ' ' ^ * \ > * 





F.igure. 2 : T^^^ining ..Programme 



Training Scs.-lot! 

1. Videot.^.pe Loctuxer-: 1 to - 9 :«iiu.it^. c,.i.-j^curf* 

2. Code first 3 minutes lecj:ur^,j^i. j:nd^2,.-^-c^--^^ 

3. Code last 5 minutes ^ lectures . 3 and, 4 (as a group) 

4 . Replay . ?.ec re j. 

5* Discuss, with codings 

6. Replay lecture 2 

7. DiscusSjWith codings \ _ . ' 
,8* Replay lecture '3 

9. Discuss, v:ith codings 

10. Replay lecture 4 - , * . - 

11, Discuss, with codings 

12 • Evaluate the recordings and the codings 
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) Corded at 6 sec. intervals 
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Plgure 3: T ime Line Display: 'Profile for 'Lectu l^e 
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Category 


Percentage Agreement 


N = 29 • ■ 
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9^^^— — ^ - 






E- ' 








Ai ' 


1 






Au 


93% 


\ 




X 


91%, 




: 


Table 1: 








Stability Coefficients for the Categories * - ^ ^ . V . 


: * ^ • 
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(ten days between two observations,, one^ . ^ 
trained observer) ;^ ' / - 










* * 






. - * " - ' 
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-Category- ■ 


* Prodiict "Momerft" 
Correlation 


J^^A8_ • — 1^ 

12 point rating scale ' 


P 






E ■ 


..82 




■ P + E. 




Ai . . - 






. In , 


\ n.a. ^ 


1 ^ ' • ^ 1 







(n.a, not, enough data) 



Table^gj - ■ . 

Concurrent Validity., of the CHegories 
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